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SUMMARY 


This paper compared groups of Internal and External CTs on Professional 


Applicant Test Battery (PATB) scores and on supervisory ratings of perform- 


ance and potential gathered by the Committee on Professional Manpower 


(Manpower Ratings). Groups of 326 External and 60 Internal male CIs who 
entered on duty during fiscal years 1%3 through 1967 were compared on 76 
PATB variables and six Manpower Ratings. Only seven statistically signifi- 
cant mean differences between the two groups of CTs emerged, and in all 
cases the absolute magnitude of the differences was not large. It was 
concluded that Internal and External CIs, on the average, do not differ to 
a significant degree on a wide variety of test performances and on super- 


visory ratings of job performance and potential. 
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A COMPARISON OF INTERNAL AND EXTERNAL CAREER 
TRAINEES ON PATB TEST SCORES AND SUPERVISORS' 
RATINGS OF CURRENT PERFORMANCE AND FUTURE 
POTENTIAL 
The purpose of this study was to compare the yerformances of Internal 

and External Career Trainees (CTs) on AES's Professional Applicant Test 
Battery (PATB), and on supervisors' ratings of job performance and 
potential obtained by the Committee on Professional Manpower. Comparisons 
of this type can help provide an answer to the frequently=-asked question 
of whether Internal and External CTs differ, on the average, in important 


characteristics or qualities of relevance to the Agency. 


METHOD 
All male CTs included in the survey of the Committee on Professional 
Manpower were included in this study. A total of 386 CTs were studied, of 
whom 326 were Externals and 60 were Internals. Females were excluded, 


since PATB contains slightly different tests (and test norms) for men and 


women. All 386 CTs entered on duty during fiscal years 1963 through 1967. 


The PATB battery of tests includes 
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Professional Manpower (hereafter referred to as Manpower Ratings). 


RESULTS 

Table 1 presents a summary of the statistically significant differences 
which obtained between External and Internal CTs on PATB test scores and 
Manpower Ratings. Out of a total of 82 variables on which comparisons 
between Internal and External CTs were made (76 PATB scores and six Man- 
power Ratings), only seven statistically significant? mean differences 
emerged; this small number of significant differences barely exceeds what 
could be expected through the operation of chance alone.” It is apparent 
from this small number and pattern of statistically significant differences 
in Table 1 that the average PATB Test scores and Manpower Ratings of Internal 
and External CIs are remarkably similar. Furthermore, without exception, the 
magnitude of the average differences between Internals and Externals that 
were obtained in the seven instances of statistical significance was quite 
small. Overall, it may be concluded that Internal and External CTs obtain, 
on the average, remarkably similar PATB test scores and supervisory evalua- 


tions of job performance and potential. 


14 statistically significant mean difference is defined as an obtained 
difference between Internal and External CTs of a magnitude such that the 
probability is 95 (or more) of 100 that the obtained difference is "true," 
i.e., not due to sampling errors or other chance factors. 


At the level of statistical significance applied in this study, the 
205 level, 4.1 is the best estimate of the number of statistically signifi- 
cant differences to be expected by chance alone when statistical tests 
are performed on 82 variables on which no actual differences exist. 
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Supervisors’ Ratings of Job Performance 

In response to a request by the Committee on Professional Manpower, 
the CIs in this study were rated by their immediate supervisors on three 
aspects of their current job performance: their overall performance; the 
quantitative aspects of their performance (i.e., how much work they get done) ; 
and the qualitative aspects of their performance (i.e., the quality of their 
work). These ratings were not shown to the individuals who were rated, nor 
were they to be included in any official records. On all three of these 


ratings of job performance, the average ratings received by Internal and 


External CIs were virtually identical. On seven-point scales (Outstanding, 


Between Outstanding and Strong, Strong, Between Strong and Proficient, 
Proficient, Adequate, and Weak), the average ratings received by Internals 
and Externals on all three measures of current job performance fell 
approximately midway in the range between "Strong" and "Between Strong and 
Proficient.” Clearly, the average rated job performances of Internal CIs 


and External CIs do not differ. 
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Supervisors’ Ratings of Future Potential 

The supervisors of the CT's in this study also produced ratings of their 
supervisees' future potential. (As with the ratings of job performance, these 
ratings were not shown to the individuals who were rated nor were they 
included in any official records. ) Three separate ratings were to be 
produced for each individual. First, the supervisor rated overall potential 
on a five-point scale (Outstanding » Above Average, Average, Below Average, 


Weak). Next, each supervisor predicted (on a "yes-no" scale) whether his 


supervisee had the potential to eventually reach a senior level (GS-15) 


position in the Agency. Finally, each supervisor predicted whether his 
supervisee would eventually attain supergrade status (GS-16) in the Agency. 
Apparently some supervisors felt they did not have enough information to make 
predictions of their supervisees' potential to achieve senior level and super- 
grade status; of the 386 CTs in this study, no predictions of potential to 
achieve senior-level status were made for nine CTs and no predictions of 
potential to achieve supergrade status were made for 25 CTs. 

On the five-point scale of overall potential, there was no significant 
difference between the average ratings received by the groups of Internal 
and External CTs. The average ratings for both groups fell slightly below 
"Above Average." There appeared to be a tendency (which just missed statis~ 
tical significance) for a slightly larger percentage of External CIs to 


receive "Outstanding" ratings of overall potential; 12% of the External 


Approved For Release 2003/04/29 : CIA-RDP84-00780R003100130014-1 


Approved For Release 2003/04/29 : CIA-RDP84-00780R003100130014-1 
SECRET 


° 


9 


but only 3% of the Internal CIs received ratings of "Outstanding." 

Despite the finding of no significant difference between Externals 
and Internals in the ratings of overall potential, a significant difference 
was obtained in the percentages of Externals and Internals rated as having 
senior-level potential. Seventy-nine per cent of the External CTs were 
rated by their supervisors as having the potential to achieve GS-153 
66% of the Internals received this rating. A similar result obtained for 
the rating of supergrade potential. Whereas 37% of the External CT's were 
seen by their supervisors as having supergrade potential, 24% of the Internal 
CIs were seen as having supergrade potential. This difference between the 
percentages of Internals and Externals rated as having supergrade potential 
failed to attain statistical significance, however. 


It thus appears that a slightly larger proportion of External CTs are 


seen by their supervisors as having the potential to advance to high levels 


(i.e., GS-15) within the Agency, despite the fact that Externals are not 


seen as doing a better job than Internals now (as reflected in the ratings of 


itis difference between the percentages of Externals and Internals rated 
as having supergrade potential, while numerically equal to the percentage 
difference between Externals and Internals rated as having senior-level poten- 
tial, did not attain statistical significance; one reason for this apparent 
inconsistency is that the rating of supergrade potential was based on fewer 
cases. As was mentioned above, while nine CTs did not receive ratings of 
seniorelevel potential, 25 were not rated on supergrade potential. A second 
reason for this apparent inconsistency had to do with the nature of the 
distribution of ratings. The overall percentage of CIs rated as having super- 
grade potential was closer to 50% than was the overall percentage rated as 
having senior=level potential; the closer to "50-50 splits" sets of data of 
this type are, the larger the differences have to be before biased reach 
statistical StgTEELCBRCES 
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overall performance) or as having greater overall potential than Internals (as 


reflected in the finding of no mean difference between Internals and Externals 


on the rating of overall potential). 


DISCUSSION 

The main conclusion of this study is that Internal and External Cis do 
not differ, on the average, on a wide variety of tests and on supervisory 
ratings of job paneiieaiee and potential. Even where statistically significant 
differences were found, the actual magnitude of these differences was never 
large; there was always a great deal of overlap of the distributions of scores 
received by Internals and Externals. And, as pointed out earlier, the number 
of statistically significant differences that were found exceeded by very little 
the number that would be expected by chance alone (due to sampling errors and 
random fluctuations in scores) if no actual differences existed between Internal 
and External CTs on any of the test measures and ratings of performance and 
potential on which the two groups were compared. 

A particularly interesting finding involved the supervisory ratings of job 
performance and potential. Internal CIs did not receive lower average ratings 
on any of the measures of current job performance or on the rating of overall 
potential, yet significantly fewer Internal CIs were seen as having the poten- 
tial to achieve sadistic’ (GS-15) status within the Agency. Although more 
than one possible explanation might be offered to explain this interesting 


phenomenon, the real reasons for it are not clear at this time. 
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Finally, it is possible that External and Internal CTs do differ on 


important dimensions which are not measured by PATB test scores or super= 


visory ratings of performance and potential. This possibility can, of course, 


never be disproven. However, it does seem a bit unlikely that important 
differences between these two groups exist in view of the number and variety 
of test variables and supervisory ratings on which they were compared. This 
seems particularly true for the Manpower Ratings of performance and potential. 
While it could be argued that Internal and External CTs might differ on certain 
psychological measures not included in PATB, or perhaps not measurable by any 
test, it is unlikely that the Manpower Ratings of job performance and potential 
could be substantially improved upon for large samples of persons. These 
ratings were obtained under quite favorable circumstances; they were made for 
research purposes only, they were ‘not to be shown to the persons being rated, 
and they were designed in such a manner as to maximally reflect the degrees 

of differences among people. They were reasonably successful in reflecting 
differences among people; much greater "spread" of ratings was obtained in 


the Manpower Ratings than in the fitness reports for the same individuals. 
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